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We report on a search for the flavor-changing neutral-current decay D° — > in pp collisions 

at yfs = 1.96 TeV using 360 pb -1 of integrated luminosity collected by the CDF II detector at 
the Fermilab Tevatron collider. A displaced vertex trigger selects long-lived D° candidates in the 
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The flavor-changing neutral current (FCNC) decay 
D° — > [i + [l| is highly suppressed in the standard 
model (SM) by Glashow-Iliopoulos-Maiani Q cancella- 
tion. Burdman, Golowich, Hewett, and Pakvasa [3] es- 
timate the branching fraction to be about 10 -18 from 
short-distance processes, increasing to about 4 x 10~ 13 
with long-distance processes. These rates are many or- 
ders of magnitude beyond the reach of the present gen- 
eration of experiments; the best published upper bound 
is 1.4 x at the 90% confidence level from Belle 

However, new physics contributions can significantly 
enhance the branching ratio. The authors of Ref. Q con- 
sider the effects on D° — > that arise from a number 
of extensions to the SM: R-parity violating SUSY, mul- 
tiple Higgs doublets, extra fermions, extra dimensions, 
and extended technicolor. Some of these scenarios could 
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increase the branching fraction to the range of 10~ 8 to 
1CP 10 , and in particular R-parity violating SUSY could 
raise it to the level of the existing experimental bound. 
Similar enhancements can occur in K and B decays, but 
charm decays are sensitive to new physics couplings in 
the up-quark sector. Golowich, Hewett, Pakvasa, and 
Petrov [5j have shown that in some new physics scenar- 
ios there is a correlation between the new physics con- 
tribution to D°-D° mixing and the branching fraction 
of D° — > /i + pT . If new physics dominates both pro- 
cesses, then the measured mixing can be used to con- 
strain B(D° — > or a measurement of both can 
shed light on the phenomenology of the new physics. 

In this paper we report on a search for D° — > fi + /J,~ 
using data corresponding to 360 pb -1 of integrated lumi- 
nosity collected by the Collider Detector at Fermilab II 
(CDF II). The 65 pb -1 of integrated luminosity compris- 
ing our previous search Q is included. We significantly 
improve the sensitivity of the new search by analyzing 
a much larger data sample, extending the muon accep- 
tance beyond the central region, identifying muons with 
a likelihood technique, and discriminating signal from b— 
hadron related background with the help of a probability 
ratio. 

The CDF II detector Q components pertinent to 
this analysis are tracking systems and muon detectors. 
The inner tracking system is composed of a silicon mi- 
crostrip detector [8| surrounded by an open-cell wire drift 
chamber (COT) |9j. These are located within a 1.4 T 
solenoidal magnetic field and measure charged particle 
momenta, p. Four layers of planar drift chambers 
detect muons with pr > 1.4GeV/c and provide coverage 
in the central pseudorapidity range \rj\ < 0.6, where pt is 
the magnitude of the momentum transverse to the beam- 
line, -q = — ln(tan6*/2), and 9 is the angle of the track 
with respect to the proton beamline. Conical sections 
of drift tubes cover the forward pseudorapidity region 
0.6 < |?7| < 1.0 for muons with p T > 2.0GeV/c. 

We determine the D° — > p + fi~ branching fraction us- 
ing the kinematically similar D° — > n + Tr~ decay as a 
reference signal, 



B(D° -> M + M _ ) = — 4^ — 



N A 



(1) 



where B(D° tt+tt-) = (1.397 ± 0.027) x 10~ 3 is the 
world average branching fraction [ll| , and are 
the numbers of D° — > n + p~ and D° — > 7r + 7r~ events 
observed, and are the combined acceptances 
and efficiencies for reconstructing dimuon and dipion D° 
decays, and is the efficiency of dimuon identification. 
Except for the requirement of muon identification and the 
assignment of final state particle mass, the event selection 
criteria are the same for both the n + n~ and n + ir~ modes. 
In the spirit of obtaining an unbiased result, we hid the 
data in the signal mass window ( "blinding" ) and fixed the 
selection criteria before revealing the data in the signal 
region ( "unblinding" ) . The ratio of acceptances times 
efficiencies A n7T /A^^ is estimated from simulated data 



samples, while the efficiency for dimuon identification, 
e MM , is determined from J/ip —> /i + data. The back- 
grounds are estimated using D° — s- K~ir + data, blinded 
D° — > /i + [iT data, and simulated samples. 

The D° data come from a sample enriched in heavy 
flavor using a silicon vertex trigger that selects events 
having displaced vertices using custom hardware pro- 
cessors [H[. The trigger selects events containing 
two oppositely charged particles reconstructed as heli- 
cal tracks formed from signals in the COT and silicon 
detectors, each with px > 2 GeV/c, and transverse mo- 
mentum sum p^ + Pt > 5.5GeV/c, where ± refer to 
the oppositely charged pair. Information from the sil- 
icon detectors is used to precisely determine the track 
positions near the beamline. Decays of particles with 
picosecond lifetimes are preferentially selected by requir- 
ing each track of the pair (the trigger tracks) to have 
an impact parameter between 120 /im and 1.0 mm with 
respect to the beamline, and the pair to be consistent 
with originating from the decay of a particle traveling a 
transverse distance L xy > 200 pm from the beamline [lj] . 
Finally, the tracks must be separated azimuthally by an 
angle 2° < | Atp\ < 90°, a range that is highly efficient for 
heavy-flavor decays while suppressing non-heavy-flavor 
backgrounds. The requirements are made first at trigger 
level, and then verified in a complete event reconstruc- 
tion. 

From the trigger sample, three non-overlapping sub- 
samples of dimuon candidates are selected because the 
muon efficiencies in the central and forward regions are 
significantly different, and independent treatment im- 
proves the sensitivity. The three subsamples are central- 
central (CC) where both tracks lie in the range |?7| < 0.6, 
central-forward (CF) where one track lies in the range 
\r/\ < 0.6 and the other in the range 0.6 < \r/\ < 1.0, and 
forward-forward (FF) where both tracks fall in the range 
0.6 < M < 1.0. 

All of the D° candidates used in the analysis come 
from D* + — > D°ir + candidates. A large fraction of re- 
constructed D° mesons come from D* decays and the 
narrow resonance width limits the phase space for ran- 
dom combinatorics. The D° candidates consist of pairs of 
oppositely charged particles, matched to trigger tracks, 
with invariant mass 1.845 < -M^ < 1.890 GeV/c 2 , where 
the tracks are assigned the muon mass. The one degree 
of freedom fit for the vertex of the track pair must have 
X 2 < 20. We select D* + —> D°tt + decays by combining a 
third track, assigned the ir + mass, with the D° candidate 
and requiring the mass difference M MM+7r — Ai^ to fall in 
the range of the resonance, 144MeV/c 2 to 147MeV/c 2 , 
where /i/i + ir refers to the combination of the track pair 
with the third track. To be considered for this analysis, 
the third track must have pr > 0.4GeV/c. About 88% 
of the selected D* decays are produced directly in the 
pp interaction (prompt) with the remainder coming from 
6-hadron decay (secondary) 

Figure [1] shows the resulting invariant mass spectrum 
for D° candidates in the CC subsample. The distribu- 



tions for the CF and FF subsamples are similar, just 
scaled by the relative acceptance. The search window, 
spanning the range around the D° mass from 1.845 
to 1.890 GeV/c 2 , is heavily populated with mass-shifted 
D — > 7r + 7r~ decays and combinatorial background. The 
more numerous D° — > K~n + decays are mass shifted 
below 1.800 GeV/c 2 , well separated from the search re- 
gion. To be considered as a D° candidate, 
both D° daughter tracks must satisfy a muon likelihood 
requirement [161 ] based on energy loss information from 
the tracker, the electromagnetic and hadronic calorime- 
ter energy deposition, and track based isolation, in ad- 
dition to the muon detector information. The additional 
information reduces the probability to misidentify pions 
and kaons as muons by about a factor of 2.5 over that 
obtained with muon detector information alone, while de- 
creasing the muon identification efficiency by about 20%. 

We estimate the muon identification efficiency per 
track as a function of transverse momentum using a sam- 
ple of J/ip — > decays collected by a trigger requir- 
ing an identified muon candidate together with a second 
track displaced from the pp beamline. The second track 
has the same characteristics as a trigger track used to 
form D° decays, in particular, no muon identification re- 
quirement, giving an unbiased sample for determining the 
muon identification efficiency. The muon likelihood re- 
quirement is found to be approximately 70% efficient for 
central muons and 40% efficient for forward muons. The 
misidentification probabilities for pions and kaons are de- 
termined as a function of track px using the large sample 
of Cabibbo favored D° — > K~tt + decays selected in the 
same manner as the signal sample, but using kaon and 
pion masses, and requiring the charge of the third track 
to be opposite to the charge of the kaon. This selects the 
Cabibbo favored decays and yields a nearly pure sample 
of kaons and pions satisfying the same requirements as 
the decay tracks in the signal sample. The probability to 
satisfy the muon likelihood requirement varies smoothly 
withp T and is about 0.5% (0.2%) for tt ± , 1.3% (0.3%) for 
K + , and 0.7% (0.3%) for K~ tracks within the central 
(forward) acceptance. 

After requiring muon identification for both tracks, the 
surviving background events fall into four categories: b- 
hadron decays with two real muons (cascade dimuons); 
6-decays with one real muon (semi-muonic B decays); 
D° — > 7r + 7r~ decays where both pions are misidenti- 
fied as muons; and combinatorial background where ran- 
dom combinations of hadrons are misidentified as muons. 
The surviving D° —> ix + n~ events are associated pre- 
dominantly with real D* decays, while the other back- 
grounds are associated with a mix of real D* decays 
and random track combinations. We find the back- 
ground contributions from semi-muonic c-hadron decays 
and D° — > K~ir + decays to be negligible. 

The dominant background is cascade dimuons from in- 
clusive B — > D/iisY — > /i + [i~vVX decays where B repre- 
sents a 6-hadron, D represents a c-hadron, and X and 
Y represent other possible decay products. We estimate 
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FIG. 1: The dimuon invariant mass distribution of events in 
the CC D° —¥ fi + fi~ subsample before applying muon identifi- 
cation. The dotted lines indicate the search window, spanning 
the mass range 1.845 GeV/c 2 to 1.890 GeV/c 2 and highly pop- 
ulated with D° — > 7r + 7r~ decays. The binned data is fitted to 
a Gaussian for the mass-shifted D° — > tt + -k~ peak plus linear 
background over the mass range 1.800 to 1.950 GeV/c 2 . 



the number of cascade dimuons and semi-muonic B de- 
cays with a misidentified hadron (B — > tt^ivX, K fj,vX , 
and pfivX) using MC simulation and applying the mea- 
sured, pt dependent, muon misidentificati on p robabili- 
ties to the hadrons. The MC uses evtgen [lTf to decay 
b- and c-hadrons, with branching fractions taken from 
pl| . and a GEANT [18] detector simulation. The MC 
sample is scaled to match the J r /ip — > fJ^fJT peak from 
B — > J/ipX — > [i + fi~X decays that pass the displaced 
vertex trigger simulation to the corresponding peak in 
the data. 

Prompt D° — > signal is separable from the dom- 

inant cascade dimuon background by the tendency for 
cascade dimuon events to point away from the pp colli- 
sion point, and the longer lifetime of 6-hadrons relative 
to D° mesons. To reduce this background contribution 
we construct a probability ratio based on two quantities: 
the impact parameter, do, of the D° candidate to the 
reconstructed pp collision point; and the significance of 
the transverse displacement, sl, of the D° candidate de- 
cay from the pp collision point, defined as the transverse 
displacement divided by its uncertainty, L xy /aL x . The 
probability ratio takes the form 



p(do,s L ) 



p s (d )p s (s L ) 



p s (d Q )p s (s L ) +p B (d )p B (si 



(2) 



where p (x) [p B (x)] are the probability density functions 
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TABLE I: Background estimates, numbers of reference mode 
events, acceptances, efficiency factors, and numbers of ob- 
served events for the CC, CF, and FF dimuon classes. The 
uncertainties are statistical and systematic uncertainties com- 
bined in quadrature. 





CC 


CF 


FF 


Cascade dimuons 


3.8 ± f. 3 


2.5 ± 1.0 


1.0 ±0.5 


Semi-^t B decays 


0.54 ±0.06 


0.13 ±0.03 


0.07 ±0.02 


Combin. bkg. 


0.04 ±0.01 


0.01 ±0.01 


< 0.01 


D° -» tt+ti-- misID 


0.53 ±0.01 


0.06 ±0.01 


0.01 ±0.01 


Total bkg. 


4.9 ± 1.3 


2.7 ± 1.0 


1.0 ±0.5 




24400 ± 200 


9620 ± 130 


6940 ±110 




0.872 ±0.005 


0.872 ± 0.005 


0.872 ± 0.005 




0.437 ± 0.003 


0.257 ±0.004 


0.161 ±0.003 


Observed events 


3 





1 



FIG. 2: The probability ratio distribution of cascade dimuon 
MC and D° -» K~tv + data. The D° -> K~tt + data have the 
same mixture of prompt production and 6-decay production 
as the reference and signal modes, but are statistically inde- 
pendent. The arrow indicates the minimum value for signal 
selection. 



for the variable, x, for D° — > fi + n~ [B — > /i + [i~vvX] de- 
cays. The probability density functions are computed 
from MC simulation. This formulation assumes that the 
input variables are uncorrelated. A correlation between 
the variables has the effect of inducing a sub-optimal 
analysis performance, but does not introduce any bias 
to the result. The probability ratio distribution is shown 
in Fig.dfor B p^yrvvX MC and D° K--K+ data. 
The D° K~ir + data consists of the same admixture 
of prompt and secondary production as the D° — > 
signal and D° — > ir + ir~ reference modes. The expected 
sensitivity of the analysis was determined (following the 
procedure described below, with the D° — > K~tt + effi- 
ciency in place of the blinded D° — s> efficiency) 
for a range of probability ratio requirements, and the 
best sensitivity is achieved for a minimum requirement 
of 0.35. This requirement keeps 87% of the signal while 
removing 75% of the cascade dimuon and semi-muonic B 
decays, and combinatorial background. The results are 
summarized in the first three rows of Table U where the 
uncertainties come from statistics. 

The D° — > 7r + 7r _ misidentification background is es- 
timated by applying the pt dependent muon misidenti- 
fication probabilities to the pion tracks in D° — > tt + it~ 
events. Combinatorial background is estimated by apply- 
ing the pt dependent muon misidentification probabili- 
ties to D° candidates with masses above the search win- 
dow, 1.890 < M(j,(j, < 4.000 GeV/c 2 , where both tracks 



fail the muon likelihood requirement, and extrapolating 
to the search window with a fitted quadratic function. 
We assume the tracks are 85% pions and 15% kaons, 
based on the assumption that 6-decays are the primary 
source of trigger tracks, and include a systematic uncer- 
tainty coming from the variation when the mixture is 
varied by 10%. The results are listed in the third and 
fourth rows of Table U where the uncertainties are the 
combined statistical and systematic uncertainties. We 
check the non-peaking background estimates by compar- 
ing the background prediction and data in the dimuon in- 
variant mass range above the search window, from 1.890 
to 4.000 GeV/c 2 . As shown in Fig. [31 good agreement 
is seen between data and the background model in the 
three subsamples. An alternative method to estimate the 
non-peaking background contribution exploits the mass 
distribution being roughly flat in the mass range from 
2.000 to 2.900 GeV/c 2 . We fit this range in data with a 
constant value, indicated by "flat rate" in Fig. [3l and ex- 
trapolate to the search window. The two methods agree 
within the uncertainties listed in Table HI 

We determine by performing a \ 2 fit with Gaus- 
sian signal plus linear background to the tt + tt~ mass dis- 
tribution and integrating the Gaussian over the search 
window. The ratio of acceptances times efficiencies 
Aw/A^ is estimated using MC simulation, where the 
dominant systematic uncertainty comes from reweight- 
ing the MC to reproduce the px and rj distributions of 
D° decays reconstructed in data. The dominant source 
of inefficiency for dipion decays relative to dimuon decays 
comes from hadronic interactions in the detector, about 
an 11% relative inefficiency. Pion decay in flight accounts 
for the remaining 2% of the relative inefficiency. To de- 
termine the effective dimuon identification efficiency, , 
we convolute the pr-dependent muon identification effi- 
ciency with the pt spectrum of pions from D° 7r + 7r~. 
The results are listed in the lower half of Table HI 
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FIG. 3: The invariant mass distribution of D° — > /i + fi can- 
didate events with the final selection criteria. 



We use a hybrid frequentist/Bayesian method to deter- 
mine the branching ratio B(D° — > and perform a 
multi-channel calculation that allows for the combination 
of the three dimuon subsamples. The calculation uses 
likelihood ratio ordering [19j], and includes uncertainties 
on input quantities (nuisance parameters) according to 
Cousins-Highland [20j . For the input quantities shown in 
Table U the sensitivity for B(D° — > n + n~) is estimated 
to be 5.2 x 1CT 7 (6.0 x 1CT 7 ) at the 90% (95%) confi- 
dence level. After unblinding, we observe 3 CC, CF, 
and 1 FF dimuon events in the search window, consis- 
tent with background expectation, and calculate a limit 
of B(D° ->/i+/j-)<2.1x 10~ 7 (3.0 x 10~ 7 ) at the 90% 
(95%) confidence level. For no signal, the probability for 
the estimated background to yield the observed number 
of events or fewer is determined to be 16%. We checked 
the result with a Bayesian algorithm used in previous 
CDF analyses [2l|. Like the frequentist calculation, the 
Bayesian calculation is multi-channel and includes nui- 
sance parameters. The estimated sensitivity with the 
Bayesian algorithm is 6.5 x 10" 7 (7.8 x 10~ 7 ) at the 90% 
(95%) credibility level. The check yields less stringent 
limits of B(D a -> fi+fj,-) < 4.3 x 10~ 7 at the 90% cred- 



ibility level and B(D° 



fj+fj, 



) < 5.3 x 10" 7 at the 



95% credibility level, as expected since Bayesian limits 
are not significantly affected by downward fluctuations 
relative to expected background [22| such as those that 
occurred in the present case. 

In conclusion, we present a search for FCNC D° — > 
ji + fi~ decays using data corresponding to 360 pb -1 of 
integrated luminosity. We observe 4 candidate events in 
the search mass window while we expect 9±2 background 
events across the 3 event classes. We set an upper bound 
on the branching ratio B(D° — > /i + /i~) < 2.1 x 10~ 7 
at the 90% confidence level and B(D° — > /i + /i~) < 
3.0 x 10 -7 at the 95% confidence level. This result su- 
persedes and improves on by a factor of ten the previous 
CDF result @. Although our limit is less stringent than 
the best published result [3| , we expect to improve it sig- 
nificantly with the analysis of the full CDF data. 
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